%------------------------------------------------------------------------------
% Copyright (c) 1991-2014, Xavier Leroy and Didier Remy.
%
% All rights reserved. Distributed under a creative commons
% attribution-non-commercial-share alike 2.0 France license.
% http://creativecommons.org/licenses/by-nc-sa/2.0/fr/
%
% Translation by Till Varoquaux, Priya Hattiangdi and Prashanth Mundkur
% Reworked by Daniel C. Buenzli
%------------------------------------------------------------------------------

\chapter{\label{sec/sockets}Modern communication: sockets}
\cutname{sockets.html}

Communication with pipes has some limitations. First, it is local to a
machine: with named pipes, communicating processes must run on the
same machine and what's more, with anonymous pipes, they must share a
common ancestor. Furthermore, pipes are not very suitable for a
particularly useful model of communication: the \emph{client-server}
model. In this model, only one program, the \emph{server}, has direct
access to a shared resource.  The other programs, the \emph{clients},
access the resource by connecting to the server. The server serializes
and controls the access to the shared resource. (Example: the
\textsc{x}-window windowing system~---~the shared resources are
the screen, the keyboard and the mouse.)

The client-server model is difficult to implement with pipes. The
major difficulty is to establish the connection between a client and
the server. With anonymous pipes, it is impossible: the server and the
client would need a common ancestor that allocated an arbitrarily
large number of pipes in advance. With named pipes, the server could
read connection requests on a particular pipe. These requests would
contain the name of another named pipe created and used by the client
to communicate with the server. The problem is to ensure the mutual
exclusion of simultaneous connection requests performed by multiple
clients.

Sockets are a generalization of pipes addressing these issues. The
client-server model is shown in figure~\ref{fig/client-server}.

\begin{myfigure}
\begin{myimage}[width="70\%"]
\begin{tikzpicture}
[port/.style={draw,fill,circle,minimum size=1mm,inner sep=0mm},
 server/.style={draw, rectangle, inner sep=2mm,rounded corners},
 client/.style={draw, ellipse}]
\node (V) at (0,4) [server] {Server $V$};
\node (Vp1) at (V.-8) [port] {};
\node (Vp2) at (V.8) [port] {};
\node (U) at (-1,2) [server] {Server $U$};
\node (Up1) at (U.-8) [port] {};
\node[below right] at (Up1.east) {(1)}; % because of babel french

\node (Up2) at (U.8) [port] {};

\node (W) at (0,0) [server] {Server $W$};
\node (Wp1) at (W.-8) [port] {};
\node (Wp2) at (W.8) [port] {};

\node (A) at (5,3) [client] {Client $A$};
\node (Ap) at (A.west) [port] {};

\node (B) at (5,1) [client] {Client $B$};
\node (Bp) at (B.west) [port] {};

\draw [<->] (Up1) to [bend right=15] node[below]{(2)} (Ap);
\draw [<->,dashed] (Up2) to [bend left=15] node[above]{(3)} (Ap);

\draw [<->] (Vp2) to [bend left=20] node[above]{(5)} (Ap);
\draw [<->] (Wp2) to [bend right=20] node[below]{(6)} (Bp);
\draw [densely dotted] (U.north) to [bend left=20] node[left]{(4)} (V.south);
\draw [densely dotted] (U.south) to [bend right=20] (W.north);
\end{tikzpicture}
\end{myimage}
\caption{Client-Server model}
\label{fig/client-server}
\end{myfigure}

\begin{enumerate}
\item
The server $U$ creates a socket $s$ on a port $p$ known to the clients and
waits for connections on it (1).
%
\item
The client $A$ creates a socket and connects to the server on the port
$p$~(2). On the server, the system allocates a new socket to
communicate privately with
the client $A$ (3). In this example, the server forks off an auxiliary
server $V$~(4), closes its connection with the client $A$ (represented by the
dashed line) and lets its child $V$ handle the connection with $A$~(5).
%
\item
The server can then accept a new client $B$, establish another
connection handled in parallel by another clone $W$~(6), and so on.
%
\item
The server can close its service by closing the file descriptor
associated with the socket $s$. After a while the system frees the
port $p$ which can then be reused, for example to install another
service.

\end{enumerate}

In the model described above, the server $U$ and the client $A$
establish a private connection (3) to communicate without interference
from other clients. For that reason, this mode of communication is
referred to as the \emph{connection-oriented} mode. If the transaction
is short the server can handle the request directly (without forking)
through the connection (3). In this case, the next client must wait
for the server to be available, either because it is handling the
connection (3), or because it explicitly manages several connections
via multiplexing.

Sockets also allow a \emph{connectionless} communication mode.  In
this mode, less frequently used, the server does not establish a
private connection with the client, but responds directly to the
client's requests. We will briefly comment on this model in
section~\ref{sec/mode-deconnecte} but in the remainder of this
chapter, we mainly describe \emph{connection-oriented} communication.

\section{Sockets}

Sockets, an extension of pipes, were introduced in \textsc{bsd}
4.2. They are now found on all Unix machines connected to a
network. Special system calls are provided to establish connections
following the client-server model; they enable local and remote
communication between processes in a (nearly) transparent way.

The \emph{communication domain} of a socket limits the processes (and
the format of their address) with which we can communicate on the
socket. Different communication domains are available, for example:
\begin{itemize}
\item the Unix domain: an address is a name in the file system
  of a machine. Communication is limited to processes
  running on that machine (like with pipes).
%
\item the Internet domain: an address is an address of a
  machine on the Internet network (addresses of the form
  \ml+129.199.129.1+, for example) and a port number on that
  machine. Communication is possible between processes running on any
  two machines connected to Internet.\footnote{ The Internet network
    is made of local networks, usually Ethernet networks, connected by
    specialized links. It connects millions of machines world-wide. In
    the Internet domain, there is no difference between programs that
    communicate with a neighboring machine, connected to the same
    Ethernet cable, and those that communicate with a machine on the other
    side of the planet, through a dozen routers and a satellite link.}
\end{itemize}

The \emph{communication type} of a socket indicates whether
communication is reliable (no loss or duplication of data) and the way
the data is sent and received (a stream of bytes, or a sequence of
packets~---~small blocks of bytes). The communication type
constrains the protocol used to transmit data. Different communication
types are available, here are three of them with their properties:
%
\begin{center}
\begin{tabular}{lll}
Type & Reliable & Data representation \\
\hline
Stream & yes & byte stream \\
Datagram & no & packets \\
Segmented packets & yes & packets \\
\hline
\end{tabular}
\end{center}
% 
The \quotes{stream} type is very similar to communication with pipes.
It is used most frequently, in particular to transmit unstructured
byte sequences (\eg{} \ml+rsh+).  The \quotes{segmented packets} type
transmits data as packets: each write delimits a packet, each read
returns at most a packet. It is well suited for message-oriented
communication. The \quotes{datagram} type is the closest to the
hardware properties of an Ethernet network: data is transmitted with
packets and there is no guarantee that they reach their
destination. It is the most economical type in terms of network
resources. Some programs use it to transmit data that is not of
crucial importance (\eg{} \ml+biff+); others, to get more network
performance but with the burden of managing data losses manually. 


%
\section{Socket creation}

The \syscall{socket} system creates a new socket:
%
\begin{codefile}{tmpunix.mli}
type socket_domain = Unix.socket_domain
type socket_type = Unix.socket_type
\end{codefile}
%
\begin{listingcodefile}{tmpunix.mli}
val $\libvalue{Unix}{socket}$ : socket_domain -> socket_type -> int -> file_descr
\end{listingcodefile}
% 
The result is a file descriptor that represents the new
socket. Initially, this descriptor is \quotes{disconnected}, 
it is not ready to accept any \ml+read+ or \ml+write+. 

The first argument is a value of type \libtype{Unix}{socket\_domain}, it
specifies the socket's communication domain:
%
\begin{mltypecases}
\begin{tabular}{@{}ll}
\ml+PF_UNIX+ & The Unix domain. \\
\ml+PF_INET+ & The Internet domain.
\end{tabular}
\end{mltypecases}
% 
The second argument, a value of type \libtype{Unix}{socket\_type},
specifies the desired communication type:
%
\begin{mltypecases}
\begin{tabular}{@{}ll}
\ml+SOCK_STREAM+ & Byte streams, reliable. \\
\ml+SOCK_DGRAM+ & Packets, unreliable. \\
\ml+SOCK_RAW+ & Direct access to the lower layers of the network. \\
\ml+SOCK_SEQPACKET+ & Packets, reliable.
\end{tabular}
\end{mltypecases}
% 
The third argument is the communication protocol to use. It is usually
\ml+0+ which selects the default protocol for the given communication
domain and type (\eg{} \textsc{udp} for \ml+SOCK_DGRAM+ or
\textsc{tcp} for \ml+SOCK_STREAM+).  Other values allow to use special
protocols, for example \textsc{icmp} (\emph{Internet Control Message
  Protocol}) used by the \ml+ping+ command to send packets which
return automatically to the sender. The numbers for these special protocols
are in the \ml+/etc/protocols+ file or in the \ml+protocols+ table of
the \textsc{nis} (\emph{Network Information Service}) database, if
any. The system call \syscall{getprotobyname} returns information about a
protocol in a portable manner: 
%
\begin{codefile}{tmpunix.mli}
type protocol_entry = Unix.protocol_entry
\end{codefile}
%
\begin{listingcodefile}{tmpunix.mli}
val $\libvalue{Unix}{getprotobyname}$ : string -> protocol_entry
\end{listingcodefile}
%
Given the name of a protocol the result is a record 
of type \libtype{Unix}{protocol\_entry}. The \ml+p_proto+ field of
this record has the protocol number.

\section{Addresses}

Several socket operations use socket addresses, represented by the variant
type \ml+sockaddr+:
\begin{lstlisting}
type $\libtype{Unix}{sockaddr}$ =
  | ADDR_UNIX of string
  | ADDR_INET of inet_addr * int
\end{lstlisting}
\ml+ADDR_UNIX f+ is an address in the Unix domain, \ml+f+ is
the name of the corresponding file in the machine's file
system. \ml+ADDR_INET (a,p)+ is an address in the Internet domain,
\ml+a+ is the Internet address of a machine and \ml+p+ a port
number on this machine.

Internet addresses are represented by the abstract type
\ml+inet_addr+. The following functions convert strings of the form
\ml+128.93.8.2+ to values of type \ml+inet_addr+, and vice versa:
%
\begin{codefile}{tmpunix.mli}
type inet_addr = Unix.inet_addr
\end{codefile}
%
\begin{listingcodefile}{tmpunix.mli}
val $\indexlibvalue{Unix}{inet\_addr\_of\_string}$ : string -> inet_addr
val $\indexlibvalue{Unix}{string\_of\_inet\_addr}$ : inet_addr -> string
\end{listingcodefile}
%

Another way to obtain internet addresses is to look them up by host name in
the \ml+/etc/hosts+ table, the \textsc{nis} database or in domain name
  servers. The system call \syscall{gethostbyname} does that. On
  modern machines, the domain name servers are consulted first and
  \ml+/etc/hosts+ is only used as a fallback but in general that may
  depend on the machine's configuration.
%
\begin{codefile}{tmpunix.mli}
type host_entry = Unix.host_entry
\end{codefile}
%
\begin{listingcodefile}{tmpunix.mli}
val $\libvalue{Unix}{gethostbyname}$ : string -> host_entry
\end{listingcodefile}
% 
The argument is the host name to look for and the result a record of
type \libtype{Unix}{host\_entry}. The \ml+h_addr_list+ field of this
record is an array of Internet addresses corresponding to the machine
(the same machine can be connected to multiple networks under
different addresses).

Regarding port numbers, the most common services are listed in the
table \ml+/etc/services+ which can be read in a portable manner using
the \syscall{getservbyname} function:
%
\begin{codefile}{tmpunix.mli}
type service_entry = Unix.service_entry
\end{codefile}
%
\begin{listingcodefile}{tmpunix.mli}
val $\libvalue{Unix}{getservbyname}$ : string -> string -> service_entry
\end{listingcodefile}
% 
The first argument is the service name (\ml+"ftp"+ for \textsc{ftp}
servers, \ml+"smtp"+ for email, \ml+"nntp"+ for news servers,
\ml+"talk"+ and \ml+"ntalk"+ for commands of that name, \etc) and the
second argument is the name of the protocol: usually \ml+"tcp"+ if the
service is using the stream connection type or \ml+"udp"+ for the
datagram type. The result of \ml+getservbyname+ is a record of type
\libtype{Unix}{service\_entry} whose \ml+s_port+ field contains the
desired number.

\begin{example} To obtain the address of the \textsc{ftp} server
\ml+pauillac.inria.fr+:
%
\begin{lstlisting}
ADDR_INET((gethostbyname "pauillac.inria.fr").h_addr_list.(0),
          (getservbyname "ftp" "tcp").s_port)
\end{lstlisting}
\enlargethispage{1\onelineskip} %% To avoid a widow
\end{example}

\pagebreak

\section{Connection to a server}

The system call \syscall{connect} establishes a connection with a
server on a socket. 
%
\begin{codefile}{tmpunix.mli}
type sockaddr = Unix.sockaddr
\end{codefile}
%
\begin{listingcodefile}{tmpunix.mli}
val $\libvalue{Unix}{connect}$ : file_descr -> sockaddr -> unit
\end{listingcodefile}
%
The first argument is a socket descriptor and the second argument is
the server's address.

Once the connection is established, calls to \indexvalue{write} on the
socket descriptor send data to the server and calls to \indexvalue{read}
receive data from the server. Sockets behave like pipes for read and
write operations. First, \ml+read+ blocks if no data is available and
can return fewer bytes than requested. Second, whenever the server
closes the connection \ml+read+ returns \ml+0+ and \ml+write+ sends a
\ml+sigpipe+ signal to the calling process.

\ml+connect+ binds the socket to a local address chosen by the
system. Sometimes, it is preferable to manually choose this address.
This can be done by calling the function \ml+bind+ (see
section~\ref{sec/service}) before \ml+connect+.

The \ml+netstat+ Unix command lists the current connections on the
machine and their status.

\section{Disconnecting sockets}

There are two ways to disconnect a socket. The first is to
call \indexvalue{close} on the socket. This closes the read and write
sides of the connection, and deallocates the socket. But sometimes
this is too brutal, for example we may want to close the connection
from the client to the server to indicate an end of file but keep the
connection open in the other direction to get remaining data from the
server. The system call \syscall{shutdown} allows to close the
connection gradually.
%
\begin{codefile}{tmpunix.mli}
type shutdown_command = Unix.shutdown_command
\end{codefile}
%
\begin{listingcodefile}{tmpunix.mli}
val $\libvalue{Unix}{shutdown}$ : file_descr -> shutdown_command -> unit
\end{listingcodefile}
%
The first argument is the descriptor of the socket to close and the
second a value of type \libtype{Unix}{shutdown\_command} indicating
which direction to close:
\begin{mltypecases}
\begin{tabular}{@{}lp{0.7\textwidth}}
\ml+SHUTDOWN_RECEIVE+ & Closes the socket for reading; \ml+write+
on the other end of the connection will send a \ml+sigpipe+ signal to
the caller. \\
%
\ml+SHUTDOWN_SEND+ & Closes the socket for writing; \ml+read+ on the
other end of the connection returns an end of file. \\
%
\ml+SHUTDOWN_ALL+ & Closes the socket for reading and writing;
unlike \ml+close+, the socket descriptor is not deallocated.
\end{tabular}
\end{mltypecases}
% 
Note that disconnecting a socket can take some time whether done with
\ml+close+ or \ml+shutdown+.


\section{\label{sec/univclient}Complete example: the universal client}

We program a \ml+client+ command such that \ml+client host port+
establishes a connection on the port \ml+port+ of the machine named
\ml+host+, sends on the resulting socket the data it reads on its
standard input and writes the data it receives on its standard output.
For instance, the command
\begin{lstlisting}
echo -e 'GET /~remy/ HTTP/1.0\r\n\r\n' | ./client pauillac.inria.fr 80
\end{lstlisting}
connects to the port \ml+80+ of \ml+pauillac.inria.fr+ and sends an
\textsc{http} request for the web page \ml+/~remy/+.

This command is a \quotes{universal} client application in the sense
that it factors out the code to establish a connection common to many
clients and delegates the implementation of the specific protocol to
the program that calls \ml+client+.

The library function \ml+Misc.retransmit fdin fdout+ reads data on the
descriptor \ml+fdin+ and writes it on \ml+fdout+. It terminates,
without closing the descriptors, when the end of file is reached on
the input descriptor. Note that \ml+retransmit+ may be interrupted by
a signal.
\begin{codefile}{misc.mli}
val retransmit : file_descr -> file_descr -> unit
(** [retransmit fdin fdout] copies the contents of fdin into fdout, until
it receives end of file in fdin. It does not close descriptors at the
end. *)
\end{codefile}
\begin{listingcodefile}{misc.ml}
let retransmit fdin fdout =
  let buffer_size = 4096 in
  let buffer = String.create buffer_size in
  let rec copy () = match read fdin buffer 0 buffer_size with
    | 0 -> ()
    | n -> ignore (write fdout buffer 0 n); copy () 
  in
  copy ();;
\end{listingcodefile}
The serious matter starts here.
\begin{listingcodefile}[style=numbers]{client.ml}
open Sys;;
open Unix;;

let client () =
  if Array.length Sys.argv < 3 then begin
    prerr_endline "Usage: client <host> <port>";
    exit 2;
  end;
  let server_name = Sys.argv.(1)
  and port_number = int_of_string Sys.argv.(2) in
  let server_addr =
    try (gethostbyname server_name).h_addr_list.(0)
    with Not_found ->
      prerr_endline (server_name ^ ": Host not found");
      exit 2 in
  let sock = socket PF_INET SOCK_STREAM 0 in
  connect sock (ADDR_INET(server_addr, port_number));
  match fork () with
  | 0 -> $\label{prog:add_signal_ignore}$
      Misc.retransmit stdin sock;
      shutdown sock SHUTDOWN_SEND;
      exit 0
  | _ ->
      Misc.retransmit sock stdout;
      close stdout;
      wait ();;

handle_unix_error client ();;
\end{listingcodefile}

We start by determining the Internet address of the machine to which
we want to connect. It can be specified by a host name or in numerical
form, \ml+gethostbyname+ correctly handles both cases. Then, we create
a socket of type stream in the Internet domain with the default protocol
and connect it to the address of the machine.

The process is then cloned with \ml+fork+. The child process copies
the data from its standard input to the socket. Once the end of
standard input is reached it closes the connection in the sending
direction and terminates. The parent process copies the data it reads
on the socket to its standard output. Once the end of file is reached
on the socket, it closes the standard output, synchronizes with the
child process and terminates.

The connection is closed either by the client or by the server:
\begin{itemize}
\item If the child client receives an end of file on its standard
  input, it closes the connection in the client to server direction
  and terminates. The server upon receiving the end of file on its
  socket should, possibly after some further processing, close the
  connection in the other direction. Thus the parent client eventually
  receives an end of file on the socket and terminates normally.
%
\item If the server closes the connection. The parent client receives
  an end of file on the socket and waits on the child. The
  child client is killed by a \ml+sigpipe+ signal the next time it
  tries to write on the socket. This does not however report that the
  connection was lost. If that is needed we can ignore
  the \ml+sigpipe+ signal by inserting the following line after
  line~\ref{prog:add_signal_ignore}:
\begin{lstlisting}
ignore (signal sigpipe Signal_ignore)
\end{lstlisting}

\noindent and the write will raise an \ml+EPIPE+ error instead.
%
\item If the parent or child client exits prematurely the socket will
  be closed for reading or writing. 
  Whenever the server detects this information, it closes its
  side of the connection, which is eventually detected by the other part of the
  client. 
\end{itemize}


\section{\label{sec/service}Establishing a service}

Having seen how a client connects to a server, we now show how a
server can provide a service for clients. First we need to associate a
particular address to a socket to make it reachable from the network.
The system call \syscall{bind} does this:
%
\begin{listingcodefile}{tmpunix.mli}
val $\libvalue{Unix}{bind}$ : file_descr -> sockaddr -> unit
\end{listingcodefile}
% 
The first argument is the socket descriptor and the second the address
to bind. The constant Internet address \libvalue{Unix}{inet\_addr\_any} can be
used to bind all the Internet addresses that the machine has (it may
be on multiple sub-networks).

We then indicate that the socket can accept connections with the
system call \syscall{listen}:
%
\begin{listingcodefile}{tmpunix.mli}
val $\libvalue{Unix}{listen}$ : file_descr -> int -> unit
\end{listingcodefile}
% 
The first argument is the socket descriptor and the second is the
number of request that can be be put on hold while the server is busy
(ranges from a few dozen to several hundreds for large servers). When
the number of waiting clients exceeds this number, additional client 
connection requests fail.

Finally, connection requests on a socket descriptor are received via the 
system call \syscall{accept}:
%
\begin{listingcodefile}{tmpunix.mli}
val $\libvalue{Unix}{accept}$ : file_descr -> file_descr * sockaddr
\end{listingcodefile}
% 
When the call returns the socket given in argument is still free and
can accept more connection request. The first component of the result
is a new descriptor connected to the client, everything written
(resp. read) on that socket can be read (resp. is written) on the
socket the client gave to \ml+connect+. The second component of the
result is the address of the client. It can be used to check that the
client is authorized to connect (for example this is what the
\textsc{x} server does, \ml+xhost+ can be used to add new
authorizations), or to establish a second connection from the server
to the client (as \textsc{ftp} does for each file transfer request).

The general structure of a \textsc{tcp} server is as follows.
%
\begin{codefile}{misc.mli}
val install_tcp_server_socket : Unix.sockaddr -> Unix.file_descr
(** [install_tcp_server_socket sockaddr] creates a socket in the internet
 domain,  binds it to the address [sockaddr], listens to it, and returns
it. *)

val tcp_server :
 (file_descr -> file_descr * sockaddr -> 'a) -> sockaddr -> unit
(** [tcp_server f addr] installs the tcp service [f] at the Internet domain
   address  [addr]. For each connection to the service, the
   function [f] receives the server's socket and the client's
   socket-address pair as parameters. It is the responsibility of  [f] to
   close the client connection when done. *)
\end{codefile}
%
\begin{listingcodefile}{misc.ml}
let install_tcp_server_socket addr =
  let s = socket PF_INET SOCK_STREAM 0 in
  try
    bind s addr;
    listen s 10;
    s
  with z -> close s; raise z;;
\end{listingcodefile}
The library function \ml+Misc.install_tcp_server addr+ creates a
socket of type stream in the Internet domain with the default protocol 
and prepares it to accept new connection requests on the address \ml+addr+
with \ml+bind+ and \ml+listen+. Given that this is a library
function, we close the socket in case of an error.

\begin{listingcodefile}{misc.ml}
let tcp_server treat_connection addr =
  ignore (signal sigpipe Signal_ignore);
  let server_sock = install_tcp_server_socket addr in
  while true do
      let client = restart_on_EINTR accept server_sock in
      treat_connection server_sock client
  done;;
\end{listingcodefile}
The library function \ml+Misc.tcp_server+ creates a socket with
\ml+install_tcp_server+ and enters an infinite loop. At each iteration
of the loop it waits for a connection request with \ml+accept+ and
treats it with the function \ml+treat_connection+. Since this is a
library function we restart the \ml+accept+ call if it is
interrupted. We also ignore the signal \ml+sigpipe+ so that unexpected
disconnection raise an \ml+EPIPE+ exception that can be caught by
\ml+treat_connection+ rather than killing the server. Note that it in
any case it is \ml+treat_connection+'s duty to close the \ml+client+
descriptor at the end of the connection.

The function \ml+treat_connection+ is also given the descriptor of the
server so that if it \ml+fork+s or or \ml+double_fork+s it can be
closed by the child.

Now suppose we have the following, application specific, \ml+service+
function that talks to the client end ends by closing the connection:
%
\begin{lstlisting}
let service (client_sock, client_addr) =
  (* Handle the client on the descriptor client_sock *)
  (* And when we are done: *)
  close client_sock;;
\end{lstlisting}
%

The server itself can treat each connection sequentially. The
following library function, in \ml+Misc+, captures this pattern:
\begin{codefile}{misc.mli}
val sequential_treatment :
file_descr -> (file_descr * sockaddr -> unit) ->
file_descr * sockaddr -> unit
(** [sequential_treatment server service client] runs [service] provided
      on server for one client. The server is given by its socket [server]
      and the client is given by its socket-address pair [client]. After
      initialization the service is performed by applying [service] to
      [client]. The treatment is sequential, that is, the function only
      returns when the service is completed. *)
\end{codefile}
%
\begin{listingcodefile}{misc.ml}
let sequential_treatment server service client = service client
\end{listingcodefile}
However as the server cannot handle any other requests while serving a client,
this scheme is only appropriate for quick services, where the
\ml+service+ function always runs in a short, bounded, amount of time (for
instance, a date server).
% L'exception \ml+EPIPE+ peut <EA>tre lev<E9>e par un client qui ferme
% pr<E9>matur<E9>ment la connexion, c'est au service de prendre cet aspect
% en compte.  qu'il convient d'ignorer: la requ<EA>te en cours est
% simplement interrompue.  Les autres types d'erreurs devront <EA>tre
% rattrap<E9>es par la fonction \ml+service+.

Most servers delegate the service to a child process: \ml+fork+ is called
immediately after \ml+accept+ returns. The child process handles the
connection and the parent process immediately retries to \ml+accept+. We
obtain the following library function in \ml+Misc+:
%
\begin{codefile}{misc.mli}
val fork_treatment :
 file_descr -> (file_descr * sockaddr -> unit) ->
 file_descr * sockaddr -> unit
(** same as [sequential_treatment] but the treatment is concurrently
 performed by a forked child process of the server. The parent process
 will have to free the child when completed. *)
\end{codefile}
%
\begin{listingcodefile}{misc.ml}
let fork_treatment server service (client_sock, _ as client) =
  let treat () = match fork () with
    | 0 -> close server; service client; exit 0
    | k -> ()
  in
  try_finalize treat () close client_sock;;
\end{listingcodefile}
% 
Note that it is essential that the parent closes \ml+client_sock+
otherwise the close made by the child will not terminate the
connection (besides the parent would also quickly run out of
descriptors). The descriptor is also closed if the fork fails, for the
server may eventually decide the error is not fatal and continue to
operate.

Similarly, the child immediately closes the \ml+server+ descriptor on
which the connection request was received. First, it does not need
it. Second, the server may stop accepting new connections before
the child has terminated. The call to \ml+exit 0+ is important since
it ensures that the child terminates after the execution of the
service and that it does not start to execute the server loop.

So far we ignored the fact that children will become zombie processes
and that we need to recover them. There are two ways to to so. The
simple approach is to have a grandchild process handle the connection
using a double fork (see page~\pageref{double-fork}). This gives the
following library function, also in \ml+Misc+:
%
\begin{codefile}{misc.mli}
val double_fork_treatment :
file_descr -> (file_descr * sockaddr -> unit) ->
file_descr * sockaddr -> unit
(** same as [fork_treatment] but the treatment is performed after a double
   fork. The forked process will be freed automatically on
   completion. *)
\end{codefile}
%
\begin{listingcodefile}{misc.ml}
let double_fork_treatment server service (client_descr, _ as client) =
  let treat () = match fork () with
    | 0 ->
        if fork () <> 0 then exit 0;
        close server; service client; exit 0
    | k ->
        ignore (restart_on_EINTR (waitpid []) k)
  in
  try_finalize treat () close client_descr;;
\end{listingcodefile}
% 
However with this approach the server loses all control on the
grandchild process. It is better to have the processes handling
services and the server in the same process group so that the whole
group can be killed at once to terminate the service. For this reason servers
usually keep the fork treatment but add children recovering code, for
example in the handler of the \ml+sigchld+ signal (see the function
\ml+Misc.free_children+ on page~\pageref{ex/childs}).

\section {Tuning sockets}

Sockets have numerous internal parameters that can be tuned: the size
of the transfer buffer, the size of the minimum transfer, the behavior
on closing, \etc{}

These parameters have different types, for this reason there are as
many \libvalue{Unix}{getsockopt} and \libvalue{Unix}{setsockopt}
\ocaml{} functions as there are types. Consult the \ocaml{}
documentation of the function \libvalue{Unix}{getsockopt} and its variants
to get a detailed list of those options and the \textsc{posix}
reference for \syscall{getsockopt} and \syscall{setsockopt} for their
exact meaning.

\begin{example} 
The following two parameters apply only to sockets of type stream in the
Internet domain.

In the \textsc{tcp} protocol, the disconnection of a socket is
negotiated and hence takes some time.  Normally a call to
\indexvalue{close} returns immediately, and lets the system
negotiates the disconnection. The code below turns \ml+close+ on the
socket \ml+sock+ into a blocking operation. It blocks either
until all the sent data has been transmitted or until 5 seconds have
passed.
%
\begin{lstlisting}
setsockopt_optint sock SO_LINGER (Some 5);;
\end{lstlisting}
%

The \ml+SO_REUSEADDR+ option allows the \indexvalue{bind} system call
to allocate a new socket on a local address immediately after the
socket \ml+sock+ bound on that address is closed (there is however the
risk to get packets intended for the old connection). This option
allows to stop a server and restart it immediately, very useful for
testing purposes.
%
\begin{lstlisting}
setsockopt sock SO_REUSEADDR;;
\end{lstlisting}
\end{example}

\section{Complete example: the universal server}

We program a \ml+server+ command such that:
%
\begin{lstlisting}
./server port cmd arg1 ... argn
\end{lstlisting}
% 
receives connection requests on the port \ml+port+ and, for each
connection, executes \ml+cmd+ with the arguments \ml+arg1 ... argn+
and the socket connection as its standard input and output. For
example, if we execute:
%
\begin{lstlisting}
./server 8500 grep foo
\end{lstlisting}
%
on the \ml+pomerol+ machine and the universal client (see
section~\ref{sec/univclient}) on an other machine as follows:
%
\begin{lstlisting}
./client pomerol 8500 < /etc/passwd
\end{lstlisting}
%
the client displays the same result as if we had typed:
%
\begin{lstlisting}
grep foo < /etc/passwd
\end{lstlisting}
%
except that \ml+grep+ is executed on \ml+pomerol+, and not on the local machine.

This command is a \quotes{universal} server in the sense that it factors
out the code common to many server and delegates the implementation of
the specific service and communication protocol to the \ml+cmd+ program it
launches.
%
\begin{listingcodefile}[style=numbers]{server.ml}
open Sys;;
open Unix;;

let server () =
  if Array.length Sys.argv < 2 then begin
    prerr_endline "Usage: client <port> <command> [arg1 ... argn]";
    exit 2;
  end;
  let port = int_of_string Sys.argv.(1) in
  let args = Array.sub Sys.argv 2 (Array.length Sys.argv - 2) in
  let host = (gethostbyname(gethostname ())).h_addr_list.(0) in $\label{prog:gethost}$
  let addr = ADDR_INET (host, port) in
  let treat sock (client_sock, client_addr as client) =
    (* log information *)
    begin match client_addr with
    | ADDR_INET(caller, _) ->
        prerr_endline ("Connection from " ^ string_of_inet_addr caller);
    | ADDR_UNIX _ ->
        prerr_endline "Connection from the Unix domain (???)";
    end;
    (* connection treatment *)
    let service (s, _) =
      dup2 s stdin; dup2 s stdout; dup2 s stderr; close s;
      execvp args.(0) args
    in
    Misc.double_fork_treatment sock service client in
  Misc.tcp_server treat addr;;

handle_unix_error server ();;
\end{listingcodefile}
% 
The address given to \ml+tcp_server+ contains the Internet address
of the machine running the program; the usual way to get it
(line~\ref{prog:gethost}) is by calling
\indexlibvalue{Unix}{gethostname}. But in general many addresses are
referencing the same machine. For instance, the address of the
\ml+pauillac+ machine is \ml+128.93.11.35+, it can also be accessed
locally (provided we are already on the \ml+pauillac+ machine) with
the address \ml+127.0.0.1+. To provide a service on all the addresses
pointing to this machine, we can use the constant Internet address
\libvalue{Unix}{inet\_addr\_any}.

The service is handled by a \quotes{double fork}. The \ml+service+
function, executed by the child, redirects standard input and the two
standard output on the connection socket and executes the requested
command (note that the handling of the service cannot be done
sequentially).

The connection is closed without any intervention of the \ml+server+
program. One of the following cases occurs:
\begin{itemize}
\item The client closes the connection in the client to server
  direction. The command launched by the server receives an end of
  file on its standard input. It finishes what it has to do, and calls
  \ml+exit+ when it is done. This closes the standard outputs which
  are the last descriptors open for writing on the connection and the
  client receives an end of file on its socket.

\item The client ends prematurely and closes the connection in the
  server to client direction. The command launched by the server may
  then receive a \ml+sigpipe+ signal the next time it tries to write
  data on the connection. This may kill the process but is perfectly
  acceptable since nothing is now reading the output of this command.

\item The command launched by the server exits before having read the
  end of file on its input.  The client receives a \ml+sigpipe+ signal
  (or an \ml+EPIPE+ exception) when it tries to write on the
  connection.
\end{itemize}

\subsection*{Precautions}

Writing a server requires more care than writing a client. While the
client usually knows the server to which it connects, the server knows
nothing about its clients and particularly if the service is public,
the client can be \quotes{hostile}. The server must therefore guard
itself against all pathological cases.

A typical attack is to open connections and leave them open without
transmitting requests. After accepting the connection the server is
blocked on the socket as long as the client stays connected. An
attacker can saturate the service by opening a lot of unused
connections. The server must be robust against these attacks: it must
only accept a limited number of simultaneous connections to avoid
system resources exhaustion and it must terminate connections that
remain inactive for too long. 

%% Il devra aussi, pouvoir continuer à servir normalement les requêtes en cours
%% indépendemment de l'attaque.

A sequential server handling connections without forking is
immediately exposed to this blocking issue. It will be unresponsive for
further request even though it does nothing. A solution for a
sequential server is to multiplex the connections, but it can be
tricky to implement. The solution with a parallel server is more
elegant, but it still needs a timeout, for example by programming an alarm
(see section~\ref{sec/usingsignals}).


\section{\label{sec/mode-deconnecte}Communication in connectionless mode}

The \textsc{tcp} protocol used by most connections of type
\ml+SOCK_STREAM+ works only in connection-oriented mode. Conversely,
the \textsc{udp} protocol used by most connections of type
\ml+SOCK_DGRAM+ always works in connectionless mode, there is
no established connection between the two machines. For this type of
sockets, data is transmitted with the system calls \syscall{recvfrom} 
and \syscall{sendto}.
%
\begin{codefile}{tmpunix.mli}
type msg_flag = Unix.msg_flag
\end{codefile}
%
\begin{listingcodefile}{tmpunix.mli}
val $\libvalue{Unix}{recvfrom}$ :
  file_descr -> string -> int -> int -> msg_flag list -> int * sockaddr

val $\libvalue{Unix}{sendto}$ :
  file_descr -> string -> int -> int -> msg_flag list -> sockaddr -> int
\end{listingcodefile}
% 
Their interface is similar to \ml+read+ and \ml+write+, they
return the size of the transferred data. The
call \ml+recvfrom+ also returns the address of the sending machine.

We can call \indexvalue{connect} on a socket of type \ml+SOCK_DGRAM+
to obtain a pseudo-connection. This pseudo-connection is just an
illusion, the only effect is that the address passed in argument is
memorized by the socket and becomes the address used for sending and
receiving data (messages coming from other addresses are ignored). It
is possible to call \ml+connect+ more than once to change the address
or disconnect the pseudo-connection by connecting to an invalid
address like \ml+0+. In contrast, doing this with a socket of type stream
would generally issue an error.

\section{Low level reads and writes}

The system calls \syscall{recv} and \syscall{send} respectively
generalize \indexvalue{read} and \indexvalue{write} but they work only 
on socket descriptors.
%
\begin{listingcodefile}{tmpunix.mli}
val $\libvalue{Unix}{recv}$ : file_descr -> string -> int -> int -> msg_flag list -> int
val $\libvalue{Unix}{send}$ : file_descr -> string -> int -> int -> msg_flag list -> int
\end{listingcodefile}
%
Their interface is similar to \ml+read+ and \ml+write+ but they add 
a list of flags of type \libtype{Unix}{msg\_flag} whose semantics is:
%
\begin{mltypecases}
\begin{tabular}{@{}ll}
\ml+MSG_OOB+ & Process out-of-band data. \\
\ml+MSG_DONTROUTE+ & Short-circuit the default routing table. \\
%
\ml+MSG_PEEK+ & Examines the data without reading it.
\end{tabular}
\end{mltypecases}
% 
These primitives can be used in connection-oriented mode instead of
\ml+read+ and \ml+write+ or in pseudo-connected mode instead of
\ml+recvfrom+ and \ml+sendto+.

\section{High-level primitives}

Examples like the universal client-server are so frequent that the
\ml+Unix+ module provides higher-level functions to establish 
or use network services.
%
\begin{listingcodefile}{tmpunix.mli}
val $\indexlibvalue{Unix}{open\_connection}$ : sockaddr -> in_channel * out_channel
val $\indexlibvalue{Unix}{shutdown\_connection}$ : Pervasives.in_channel -> unit
\end{listingcodefile}
% 
The \ml+open_connection+ function opens a connection to the given address
and creates a pair of \ml+Pervasives+ input/output channels on the resulting
socket. Reads and writes on these channels communicate with the server
but since the output channel is buffered we must flush it to ensure
that a request has been really sent. The client can shutdown the
connection abruptly by closing either of the channels (this will close
the socket) or more \quotes{cleanly} by calling
\ml+shutdown_connection+. If the server closes the connection, the
client receives an end of file on the input channel.

A service can be established with the \ml+establish_server+ function.
%
\begin{listingcodefile}{tmpunix.mli}
val $\indexlibvalue{Unix}{establish\_server}$ :
  (in_channel -> out_channel -> unit) -> sockaddr -> unit
\end{listingcodefile}
% 
The function \ml+establish_server f addr+ establishes a service on the
address \ml+addr+ and handles requests with the function \ml+f+. Each
connection to the server creates a new socket and forks. The child
creates a pair of \ml+Pervasives+ input/output channels on the socket
to communicate with the client and gives them to \ml+f+ to provide the
service. Once \ml+f+ returns the child closes the socket and exits. If
the client closes the connection cleanly, the child gets and end of
file on the input channel and if it doesn't it may receive a
\ml+sigpipe+ signal when \ml+f+ writes on the output channel.  As for the
parent, it has probably already handled another request! The
\ml+establish_server+ function never terminates, except in
case of error (\eg{} of the {\ocaml} runtime or the system during
the establishment of the service).

\section{Examples of protocols}

In simple cases (\ml+rsh+, \ml+rlogin+, \ldots), the data transmitted
between a client and a server is naturally represented by two streams
of bytes, one from the client to the server and the other in the
reverse direction. In other cases, the data to transmit is more
complex, and requires to be encoded and decoded to/from the streams of
bytes. The client and the server must then agree on a precise
transmission protocol, which specifies the format of requests and
responses exchanged on the connection. Most protocols used by Unix
commands are specified in documents called \quotes{\textsc{rfc}}
(request for comments): these documents start as proposals open
for discussion, and gradually become standards over time, as users
adopt the described protocol.\footnote{\textsc{rfc}s are available
  via anonymous \textsc{ftp} on numerous sites. In France:
  \href{ftp://ftp.inria.fr}{\texttt{ftp.inria.fr}}, in the directory
  \href{ftp://ftp.inria.fr/pub/rfc/}{\texttt{rfc}}. The reference site
  is \url{http://www.faqs.org/rfcs/}. }


\subsection*{\quotes{Binary} protocols}

Most binary protocols transmit data in a compact format, as close as
possible to the in-memory representation, in order to minimize the
encoding/decoding work needed for transmission and save network
bandwidth. Typical examples of protocols of this type are the
\textsc{x}-window protocol, which governs exchanges between the
\textsc{x} server and \textsc{x} applications, and the
\textsc{nfs} protocol (\rfc{1094}).

Binary protocols usually encode data as follows. An integer or
floating point number is represented by its 1, 2, 4, or 8 bytes binary
representation. A string by its length as an integer followed by its
contents as bytes. A structured object (tuple, record) by the
representation of its fields in order. A variable size structure
(array, list) by its length as an integer followed by the
representation of its elements. If the exact type of data being
transmitted in known to a process it can easily recreate it in its
memory. When different type of data is exchanged on a socket the data
encoding can be preceded by an integer to identify the data that
follows.


\begin{example}
The \ml+XFillPolygon+ call of the \textsc{x} library, which draws and
fills a polygon, sends a message with the following structure to the  
\textsc{x} server:
\begin{itemize}
\item the byte 69 (the code of the \ml+FillPoly+ command)
\item a padding byte
\item a 16 bit integer specifying the number $n$ of vertices of the polygon
\item a 32 bit integer identifying the window on which to draw
\item a 32 bit integer identifying the \quotes{graphic context}
\item a \quotes{form} byte, indicating whether the polygon is convex, \etc
\item a byte indicating whether the coordinates of the vertices are absolute or
  relative
\item $4n$ bytes encoding the coordinates of the polygon's vertices
  by two 16 bit integers
\end{itemize}
\enlargethispage{1\onelineskip} %% To avoid a widow
\end{example}
\pagebreak

With binary protocols we must pay attention to the computer
architecture of the communicating machines. In particular for
multi-byte integers, \emph{big-endian} machines store the most
significant byte first (that is, in memory, at the lower-address) and
\emph{little-endian} machines store the least significant byte
first. For instance, the 16 bit integer $12345 = 48 \times 256 + 57$
is represented by the byte $48$ at the address $n$ and the byte $57$
at the address $n+1$ on a big-endian machine, and by the byte
$57$ at the address $n$ and the byte $48$ at the address $n+1$ on a
little-endian machine. Hence protocols must precisely specify which
convention they use when multi-bytes integers are transmitted. Another
option is to allow both and have it specified in the header of the
transmitted message. 

The {\ocaml} system helps to encode and decode data structures (a
procedure called
\emph{marshalling}, \emph{serialization} or \emph{pickling} in the
literature) by providing two functions to convert an {\ocaml} value
into a sequence of bytes and vice versa:
%
\begin{listingcodefile}{tmppervasives.mli}
val $\indexlibvalue{Pervasives}{output\_value}$ : out_channel -> 'a -> unit
val $\indexlibvalue{Pervasives}{input\_value}$ : in_channel -> 'a
\end{listingcodefile}
% 
These function are defined to save values to a disk file and get
them back but they can also be used to transmit any value on a pipe or
a socket. They handle any {\ocaml} values except functions, preserve
sharing and circularities inside values and work correctly between
machines of different endianness. More information can be found in the
\libmodule{Marshal} module.

Note that semantically, the type of \ml+input_value+ is incorrect. It
is too general, it is not true that the result of \ml+input_value+ is
of type \ml+'a+ for any type \ml+'a+. The value returned by
\ml+input_value+ belongs to a precise type, and not to all possible
types. But this type cannot be determined at compile time, it depends
on the content of the channel read at runtime. Type-checking
\ml+input_value+ correctly requires an extension to the ML language
known as dynamic objects: values are paired with a representation of
their type allowing to perform runtime type checks. Consult
\cite{Dynamiques} for a detailed presentation.



\begin{example}
If the \textsc{x}-window protocol was written in {\ocaml}, we would
define a variant type \ml+request+ for requests sent to the
server and a \ml+reply+ type for server responses:
%
\begin{lstlisting}
type request =
  | FillPolyReq of (int * int) array * drawable * graphic_context
                                    * poly_shape * coord_mode
  | GetAtomNameReq of atom
  | ...
and reply =
  | GetAtomNameReply of string
  | ...
\end{lstlisting}
%
The core of the server would be a loop that reads and decodes a
request and responds by writing a reply:
%
\begin{lstlisting}
(* Get a connection request on the descriptor s *)
let requests = in_channel_of_descr s
and replies  = out_channel_of_descr s in
try
  while true do
    match input_value requests with
        ...
      | FillPoly(vertices, drawable, gc, shape, mode) ->
          fill_poly vertices drawable gc shape mode
      | GetAtomNameReq atom ->
          output_value replies (GetAtomNameReply(get_atom_name atom))
      | ...
  done
with End_of_file -> (* end of the connection *)
\end{lstlisting}
%
The functions of the \textsc{x} library, 
linked with each application would have the following structure:
%
\begin{lstlisting}
(* First establish a connection with the server on the descriptor s *)
...
let requests = out_channel_of_descr s
and replies  = in_channel_of_descr s;;

let fill_poly vertices drawable gc shape mode =
  output_value requests
               (FillPolyReq(vertices, drawable, gc, shape, mode));;
let get_atom_name atom =
  output_value requests (GetAtomNameReq atom);
  match input_value replies with
  | GetAtomNameReply name -> name
  | _ -> fatal_protocol_error "get_atom_name";;
\end{lstlisting}
\end{example}


\paragraph{Remote procedure call}

Another typical incarnation of binary protocols is remote procedure
calls (\textsc{rpc}). A user on machine A wants to call a function
\ml+f+ on a machine B. This is obviously not directly possible. It can
be programmed on a case by case basis using the system to open
a connection to the machine B, execute the call and send the
result back to the machine A.

\begin{myfigure}
\begin{myimage}[width="80\%"]
\begin{tikzpicture}
[box/.style={draw, rectangle, inner sep=2mm,rounded corners}]

\node (Sys1) at (0,0) [box, minimum width=30mm] {System};
\node (Rpc1) at (0,1.5) [box] {RPC Server};
\node (User1) at (1,2.75) [box] {User 1};
\node (User2) at (-1,3.5) [box] {User 2};
\node (M1label) at (0,4.5) {Machine A};
\node (M1) [box,fit=(Sys1)(Rpc1)(User1)(User2)(M1label)] {};
\draw [<->] (Sys1) to (Rpc1);
\draw [<->] (Rpc1) to (User1);
\draw [<->] (Rpc1) to (User2);

\node (Sys2) at (5.5,0) [box, minimum width=30mm] {System};
\node (Rpc2) at (5.5,1.5) [box] {RPC Server};
\node (Callf) at (4.5,2.75) [box] {Call f};
\node (Callg) at (6.5,3.5) [box] {Call g};
\node (M2label) at (5.5,4.5) {Machine B};
\node (M2) [box,fit=(Sys2)(Rpc2)(Callf)(Callg)(M2label)] {};
\draw [<->] (Sys2) to (Rpc2);
\draw [<->] (Rpc2) to (Callf);
\draw [<->] (Rpc2) to (Callg);

\draw [<->] (Sys1.east) to (Sys2.west);
\draw [<-, dashed] (User1) to [bend right=8] (Callf);
\draw [->, dashed] (User1) to [bend left=8] (Callf);
\draw [<-, dashed] (User2) to [bend right=5] (Callg);
\draw [->, dashed] (User2) to [bend left=5] (Callg);
\end{tikzpicture}
\end{myimage}
\caption{Remote procedure call}
\label{fig/rpc}
\end{myfigure}

But since this is a common situation, an \textsc{rpc} service can
handle that (see figure~\ref{fig/rpc}). An \textsc{rpc} server runs on
both machine A and B. A user on machine A requests the \textsc{rpc}
server on the machine to execute a function on the distant machine
B. The server on A relays the request to the \textsc{rpc} server on
machine B which executes the call to \ml+f+, sends the result back to
the server on A which gives the result to the user. The point is that
another user can call another function on B by going through the same
server on A. The connection work is shared by the \textsc{rpc} service
installed on the machines A and B and from the perspective of the
users, everything happens as if these calls were simple function calls
(dashed arrows).

\subsection*{\quotes{Text} protocols}

Network services where the efficiency of the protocol is not crucial
are often \quotes{text} protocols. A \quotes{text} protocol is in fact a
small command language. Requests are command lines, the first word
identifies the request type and the possible remaining words the
command's arguments.  Responses are also made of one or more lines of
text, often starting with a numerical code to identify the kind of
response. Here are some \quotes{text} protocols:
\begin{center}
\begin{tabular}{lll}
Name & Descr. & Purpose \\
\hline
\textsc{smtp} (Simple Mail Transfer Protocol)
&\rfc{821}
&Electronic mail
\\
\textsc{ftp}
(File Transfer Protocol)
&\rfc{959}
&File transfer
\\
\textsc{nttp}
(Network News Transfer Protocol)
&\rfc{977}
&News reading
\\
\http/1.0 (HyperText Transfer Protocol)
& \rfc{1945}
&Web navigation
\\
\http/1.1 (HyperText Transfer Protocol)
& \rfc{2068}
&Web navigation\smallskip\\
\hline
\end{tabular}
\end{center}
The great advantage of these protocols is that the exchanges between
the server and the client are human readable. For example we can just
use the \ml+telnet+ command to talk directly to the server. Invoke
\ml+telnet host service+ where \ml+host+ is the host name on which the
server is running the service \ml+service+ (\eg{} \ml+http+, \ml+smtp+,
\ml+nntp+, \etc) and then type in the requests as a client would, the
server's responses will be printed on standard output. This makes it easier to
understand the protocol. However coding and decoding requests and
responses is more involved than for binary protocols and the message
size also tends to be larger which is less efficient.


\begin{example}
Here is an example of an interactive dialog, in the shell, to send an
email on an \textsc{smtp} server. The
lines preceded by~\ifhtmlelse{>>}{$\rightarrow$} go from the client to the
server, and are typed in by the user. The lines preceded
by~\ifhtmlelse{<<}{$\leftarrow$} go from the server to the client.
{\ifnothtml{%
   \lstset{language={},literate={{<<}{{$\leftarrow$}}2 {>>}{{$\rightarrow$}}2}}}
\begin{lstlisting}
    telnet margaux smtp
    Trying 128.93.8.2 ...
    Connected to margaux.inria.fr.
    Escape character is '^]'.
<<  220 margaux.inria.fr Sendmail 5.64+/AFUU-3 ready at Wed, 15 Apr 92 17:40:59
>>  HELO pomerol.inria.fr
<<  250 Hello pomerol.inria.fr, pleased to meet you
>>  MAIL From:<god@heavens.sky.com>
<<  250 <god@heavens.sky.com>... Sender ok
>>  RCPT To:<xleroy@margaux.inria.fr>
<<  250 <xleroy@margaux.inria.fr>... Recipient ok
>>  DATA
<<  354 Enter mail, end with "." on a line by itself
>>  From: god@heavens.sky.com (Himself)
>>  To: xleroy@margaux.inria.fr
>>  Subject: Hello!
>>
>>  Is everything ok down there?
>>  .
<<  250 Ok
>>  QUIT
<<  221 margaux.inria.fr closing connection
    Connection closed by foreign host.
\end{lstlisting}}
The commands \ml+HELO+, \ml+MAIL+ and \ml+RCPT+ respectively send to
the server: the name of the client machine, the address of the sender
and the address of the recipient. The \ml+DATA+ command asks to send
the body of the email. The body of the message is then entered and
ended by a line containing the single character \ml+'.'+ (would the body
of the email contain such a line, we just double the initial \ml+'.'+ on
that line, this additional period is then suppressed by the server). 

The responses from the server are all made of a 3 digit numerical code
followed by a comment. Responses of the form \ml+5xx+ indicate an
error and those with \ml+2xx+, that everything is fine. When the
client is a real program it only interprets the response code, the comment is
only to help the person who develops the mail system.
\end{example}

\section{Complete example: {\normalfont \http} requests}

The {\http} protocol (HyperText Transfer Protocol) is primarily used
to read documents over the famous \quotes{world wide web}. This domain
is a niche area of client-server examples: between the client that
reads a page and the server that writes it there is a myriad of
intermediary relays that act as virtual servers for the real client or
delegated clients for the real server. These relay often provide
additional service like caching, filtering, \etc.

There are several versions of the {\http} protocol. To allow us to
focus on the essentials, namely the architecture of clients or relays,
we use the simple protocol inherited from the very first versions of
the protocol. Even if dust-covered it is still understood by most
servers. At the end of the section we describe a more modern, but also
more complex, version which is needed to make real tools to explore
the web. We do however leave the translation of the examples to this
new version as an exercise. 

Version 1.0 of the {\http} protocol specified in \rfc{1945} defines
simple requests of the form:
\begin{lstlisting}
GET $\textit{sp}$ $\textit{uri}$ $\textit{crlf}$
\end{lstlisting}
where \textit{\ml+sp+} represents a space and \textit{\ml+crlf+} the
string \ml+"\r\n"+ (\quotes{return} followed by
\quotes{linefeed}). The response to a simple request is also
simple: the content of the {\URL} is sent directly, without any
headers and the end of the request is signaled by the end of file,
which closes the connection. This form of request, inherited from
version 0.9 of the protocol, limits the connection to a single 
request.

\subsection*{Fetching a {\normalfont\URL}}

We write a \ml+geturl+ program that takes a single argument, a {\URL},
retrieves the resource it denotes on the web and displays it. 

The first task is to parse the {\URL} to extract the name of the
protocol (here, necessarily \ml+"http"+), the address of the server,
the optional port and the absolute path of the document on the
server. This is done with \libmodule{Str}, {\ocaml}'s regular
expression library.
%
\begin{listingcodefile}{url.ml}
open Unix;;

exception Error of string
let error err mes = raise (Error (err ^ ": " ^ mes));;
let handle_error f x = try f x with Error err -> prerr_endline err; exit 2

let default_port = "80";;

type regexp = { regexp : Str.regexp; fields : (int * string option) list; }

let regexp_match r string =
  let get (pos, default) =
    try Str.matched_group pos string
    with Not_found ->
      match default with Some s -> s | _ -> raise Not_found in
  try
    if Str.string_match r.regexp string 0 then
      Some (List.map get r.fields)
    else None
  with Not_found -> None;;

let host_regexp =
  { regexp = Str.regexp "\\([^/:]*\\)\\(:\\([0-9]+\\)\\)?";
    fields = [ 1, None; 3, Some default_port; ] };;

let url_regexp =
  { regexp = Str.regexp "http://\\([^/:]*\\(:[0-9]+\\)?\\)\\(/.*\\)";
    fields = [ 1, None; 3, None ] };;

let parse_host host = match regexp_match host_regexp host with
  | Some (host :: port :: _) -> host, int_of_string port
  | _ -> error host "Ill formed host";;

let parse_url url = match regexp_match url_regexp url with
  | Some (host :: path :: _) -> parse_host host, path
  | _ -> error url "Ill formed url";;
\end{listingcodefile}
% 
Sending a simple request is a trivial task, as the following function
shows.
%
\begin{listingcodefile}{url.ml}
let send_get url sock =
  let s = Printf.sprintf "GET %s\r\n" url in
  ignore (write sock s 0 (String.length s));;
\end{listingcodefile}
% 
Note that the {\URL} can be complete, with the address and port of the
server, or just contain the requested path on the server.

Reading the response is even easier, since only the document is
returned, without any additional information. If there's an error in
the request, the error message returned by the server as an
\textsc{html} document. Thus we just print the response with the
function \ml+Misc.retransmit+ without indicating whether this is an
error or the desired document. The rest of the program establishes the
connection with the server.
%
\begin{listingcodefile}{url.ml}
let get_url proxy url fdout =
  let (hostname, port), path = match proxy with
    | None -> parse_url url
    | Some host -> parse_host host, url
  in
  let hostaddr =
    try inet_addr_of_string hostname
    with Failure _ ->
      try (gethostbyname hostname).h_addr_list.(0)
      with Not_found -> error hostname "Host not found"
  in
  let sock = socket PF_INET SOCK_STREAM 0 in
  Misc.try_finalize
    begin function () ->
      connect sock (ADDR_INET (hostaddr, port));
      send_get path sock;
      Misc.retransmit sock fdout
    end ()
    close sock;;
\end{listingcodefile}
%
We conclude, as usual, by parsing the command line.
%
\begin{codefile}{geturl.ml}
open Unix
open Url
\end{codefile}
%
\begin{listingcodefile}{geturl.ml}
let geturl () =
  let len =  Array.length Sys.argv in
  if len < 2 then
    error "Usage:" (Sys.argv.(0) ^ " [ proxy [:<port>] ] <url>")
  else
    let proxy, url =
      if len > 2 then Some Sys.argv.(1), Sys.argv.(2) else
      None, Sys.argv.(1)
    in
    get_url proxy url stdout;;

handle_unix_error (handle_error geturl) ();;
\end{listingcodefile}

\subsection*{\label{sec/relay}{\normalfont\http} relay}

We program an {\http} relay (or \emph{proxy}), which is a server 
that redirects {\http} requests from a client to another server
(or relay\ldots) and forwards responses from that server back to the 
client. 

The role of a relay is shown in in figure~\ref{fig/relais}.  When a
client uses a relay, it addresses its requests to the relay rather
than to the individual {\http} servers located
around the world. A relay has multiple advantages. It can store the
responses to the most recent or frequent requests and serve them without
querying the remote server (\eg{} to avoid network overload or if
the server is down). It can filter the responses (\eg{} to remove
advertisements or image, \etc). It can also simplify the development
of a program by making it see the whole world wide web through a
single server.

\begin{myfigure}
\begin{myimage}[width="70\%"]
\begin{tikzpicture}[
 server/.style={draw, rectangle, inner sep=2mm,rounded corners},
 client/.style={draw, ellipse}]
\node (Pauillac) at (0,2.5) [server] {pauillac.inria.fr};
\node (Inria) at (0,0) [server] {www.inria.fr};
\node (Kelen) at (5,3) [server] {kelen.polytechnique.fr};
\node (Relay) at (2.5,1) [server] {Relay};

\node (Client1) at (5.5,0) [client] {Browser};
\node (Client2) at (6,1.5) [client] {Other client};

\draw [<->] (Pauillac) to (Relay);
\draw [<->] (Inria) to (Relay);
\draw [<->] (Kelen) to (Relay);
\draw [<->] (Client1) to (Relay);
\draw [<->] (Client2) to (Relay);
\end{tikzpicture}
\end{myimage}
\caption{{\http} relay}
\label{fig/relais}
\end{myfigure}

The \ml+proxy port+ command launches the server on the port \ml+port+
(or if omitted, on the default port for {\http}). We reuse the code of
the \ml+get_url+ function (we assume that the functions above are
available in a \ml+Url+ module). It only remains to write the code to
analyze the requests and set up the server.
%
\begin{listingcodefile}{proxy.ml}
open Unix
open Url

let get_regexp =
  { regexp = Str.regexp "^[Gg][Ee][Tt][ \t]+\\(.*[^ \t]\\)[ \t]*\r";
    fields = [ 1, None ] }

let parse_request line = match regexp_match get_regexp line with
  | Some (url :: _) -> url
  | _ -> error line "Ill formed request"
\end{listingcodefile}
%
We establish the service with the \ml+establish_server+
function, thus we just need to define the function to handle a 
connection:
%
\begin{listingcodefile}{proxy.ml}
let proxy_service (client_sock, _) =
  let service () =
    try
      let in_chan = in_channel_of_descr client_sock in
      let line = input_line in_chan in
      let url = parse_request line in
      get_url None url client_sock
    with End_of_file ->
      error "Ill formed request" "End_of_file encountered"
  in
  Misc.try_finalize
    (handle_error service) ()
    close client_sock
\end{listingcodefile}
\label{page/get_url}
%
and the rest of the program just establishes the service:
%
\begin{listingcodefile}{proxy.ml}
let proxy () =
  let http_port =
    if Array.length Sys.argv > 1 then
      try int_of_string Sys.argv.(1)
      with Failure _ -> error Sys.argv.(1) "Incorrect port"
    else
      try (getservbyname "http" "tcp").s_port
      with Not_found -> error "http" "Unknown service"
  in
  let treat_connection s = Misc.double_fork_treatment s proxy_service in
  let addr = ADDR_INET(inet_addr_any, http_port) in
  Misc.tcp_server treat_connection addr;;

handle_unix_error (handle_error proxy) ();;
\end{listingcodefile}

\subsection*{The {\normalfont \http/1.1} protocol}

Simple {\http} requests need one connection per request. This is
inefficient because most requests on a server are followed by others
(\eg{} if a client gets a web page with images, it will
subsequently request the images) and the time to establish a
connection can easily exceed the time spent in handling the request
itself (chapter~\ref{sec/coprocessus} show how we can reduce this by
handling the requests with threads rather than processes). Version 1.1
of the {\http} described in \rfc{2068} uses complex requests that
allow to make multiple requests on a single connection\footnote{The \http/1.0
  protocol already  allows complex requests but we prefer to describe the 
  \http/1.1 protocol which deals exclusively with these.}. 

In complex requests, the server precedes every response with a header
describing the format of the response and possibly the size of the
document transmitted. The end of the document is no longer indicated
by an end of file, since we know its size. The connection can
therefore stay
open to handle more requests. Complex requests have the following form:
%
\begin{lstlisting}
GET $\textit{sp}$ $\textit{uri}$ $\textit{sp}$ HTTP/1.1 $\textit{crlf}$
$\textit{header}$ $\textit{crlf}$
\end{lstlisting}
%
The \textit{\ml+header+} part defines a list of key-value fields with
the following syntax:
%
\begin{lstlisting}
$\textit{field}$ : $\textit{value}$ $\textit{crlf}$
\end{lstlisting}
% 
Superfluous spaces are allowed around the \ml+':'+ separator and any
space can always be replaced by a tab or a sequence of spaces. The
header fields can also span several lines: in this case, and in this
case only, the \textit {\ml+crlf+} end of line lexeme is immediately
followed by a space \textit{\ml+sp+}. Finally, uppercase and lowercase
letters are equivalent in the keyword of fields and in the values of
certain fields.

Mandatory and optional fields depend on the type of request. For
instance, a \ml+GET+ request must have a field indicating the
destination machine:
%
\begin{lstlisting}
Host : $\textit{hostname}$ $\textit{crlf}$
\end{lstlisting}
%
For this type of request, we may also request, using the optional field
\texttt{If-Modified}, that the document be returned only if it has been
modified since a given date.
%
\begin{lstlisting}
If-Modified : $\textit{date}$ $\textit{crlf}$
\end{lstlisting}
%
The number of fields in the \textit{\ml+header+} is not fixed in
advance but indicated by the end of the header: a line
containing only the characters \textit{\ml+crlf+}.

Here is a complete request (on each line an implicit \ml+\n+ follows
the \ml+\r+):
%
\begin{lstlisting}
GET /~remy/ HTTP/1.1\r
Host:pauillac.inria.fr\r
\r
\end{lstlisting}
%
A response to a complex request is also a complex response. It contains
a status line, a header, and the body of the response, if any.
%
% The original showed a HTTP/1.0 response, when a HTTP/1.1 response was probably intended.
\begin{lstlisting}
HTTP/1.1 $\textit{sp}$ $\textit{status}$ $\textit{sp}$ $\textit{message}$ $\textit{crlf}$
$\textit{header}$ $\textit{crlf}$
$\textit{body}$
\end{lstlisting}
%
The fields of a response header have a syntax similar to that of a
request but the required and optional fields are different (they
depend on type of request and the status of the response~---~see the
full documentation of the protocol).

The \textit{\ml{body}} of the response can be transmitted in a
single block, in chunks or be empty: 

\begin{itemize}
\item If the body is a single block the header
contains a \ml+Content-Length+ field specifying in decimal
\textsc{ascii} notation the number of bytes in the body. 

\item If the body is transmitted in chunks, 
  the header contains a \ml+Transfer-Encoding+
field with the value \quotes{\ml+chunked+}. The body is then a set of chunks
and ends with an empty chunk.  A chunk is of the form:
%
\begin{lstlisting}
$\textit{size}$ $\textit{[ ; \textit{arg} ]}$ $\textit{crlf}$
$\textit{chunk}$ $\textit{crlf}$
\end{lstlisting}
% 
where \textit{\ml+size+} is the size of the chunk in hexadecimal
notation and \textit{\ml+chunk+} is a chunk of the response body of
the given size (the part between \textit{\ml+[+} and \textit{\ml+]+}
is optional and can safely be ignored).  The last, empty, chunk is
always of the following form:
%
\begin{lstlisting}
0 $\textit{crlf}$
$\textit{header}$ $\textit{crlf}$ $\textit{crlf}$
\end{lstlisting}
%
\item If the response does not contain a \ml+Content-Length+ field and
  that it is not chunked, the body is empty (for instance, a
response to a request of type \ml+HEAD+ contains only a header). 
\end{itemize}
%
Here is an example of a single block response:
\begin{lstlisting}
HTTP/1.1 200 OK\r
Date: Sun, 10 Nov 2002 09:14:09 GMT\r
Server: Apache/1.2.6\r
Last-Modified: Mon, 21 Oct 2002 13:06:21 GMT\r
ETag: "359-e0d-3db3fbcd"\r
Content-Length: 3597\r
Accept-Ranges: bytes\r
Content-Type: text/html\r
\r
<html>
...
</html>
\end{lstlisting}
The status \ml+200+ indicates that the request was successful. A
\ml+301+ means the {\URL} was redirected to another {\URL} defined in
the \ml+Location+ field of the response. The \ml+4XX+ statuses
indicate errors on the client side while \ml+5XX+ errors on the server
side.

\begin{exercise}[noanswer]
\label{ex/proxy-1.1}
Write a relay that works with the \http/1.1 protocol.
\end{exercise}

\begin{exercise}[noanswer]
Add a cache to the relay. Pages are saved on the hard drive and when a
requested page is available in the cache, it is served unless too old.
In that case the server is queried again and the cache updated.
\end{exercise}

\begin{exercise}[noanswer]
\label{ex/wget}
Write a program \ml+wget+ such that \ml+wget u1 u2 ... un+ makes the
requests \ml+u1+, \ml+u2+, \ldots, \ml+un+ and saves the responses in
the files \ml+./m1/p1+, \ml+./m2/p2+, \ldots, \ml+./mn/pn+ where
\ml+mi+ and \ml+pi+ are respectively the name of the server and the
absolute path of the request \ml+ui+. The program should take
advantage of the protocol to establish a single connection to a
machine \ml+m+ when it is the same for consecutive requests. 
Permanent {\URL} redirections
should be followed. The following options can also be added:
\begin{mltypecases}
\begin{tabular}{@{}lp{0.8\textwidth}}
\ml+-N+ & Do not download the {\URL} if the file \ml+./mi/ui+
does not exist or is older than the {\URL}. \\
\ml+-r+ & Download recursively all the {\URL}s embedded in the
responses which are documents in \textsc{html} format.
\end{tabular}
\end{mltypecases}
\end{exercise}


%%%% This is commented in the french version.

%% \begin{answer}
%% Il n'y a pas de difficulté particulière dans cette exercice, mais
%% l'analyse des réponses doit être méticuleuse, car le non respect du
%% protocole peut produite une perte de synchronisation entre le serveur
%% et le client qui ne se comprennent plus. De plus, ces erreurs peuvent
%% être difficile à découvrir, car la plupart des serveurs fournissent
%% des réponses à un format \quotes{standard} alors que le protocole
%% permet plus de flexibilité.

%% Le formattage des requêtes quant à lui est immédiat.
%% %
%% \begin{listingcodefile}{wgetlib.ml}
%% open Unix
%% open Url
%% let http_version = "HTTP/1.1";;
%% let send chan header =
%%   let send_line line =
%%     output_string chan (String.concat " " line);
%%     output_string chan "\r\n" in
%%   List.iter send_line header;
%%   output_string chan "\r\n";
%%   flush chan;;

%% let send_get chan hostname path =
%%   send chan
%%     [ [ "GET"; path; http_version; ];
%%       ["Host:"; hostname; ];
%%     ];;
%% \end{listingcodefile}
%% %
%% L'analyse des entêtes des réponses est une partie facile mais peu
%% intéressante. On laisse au lecteur le soin d'écrire des fonctions
%% d'analyse:
%% %
%% \begin{lstlisting}
%% val parse_command : in_channel -> (string * string) list
%% val parse_response : in_channel -> (int * string) * (string * string) list
%% val parse_chunk : in_channel -> int * (string * string) list
%% val parse_crlf : in_channel -> unit
%% val tokenize : string -> string list
%% \end{lstlisting}
%% %
%% L'appel \ml+parse_header chan+ lit une suite de champs (les noms
%% champs de champs sont transformés en lettre minuscules).  L'appel
%% \ml+parse response chan+ lit une réponse complète retourne le statut
%% de la réponse (entier), le message et la liste de champs.  L'appel
%% \ml+parse chunk chan+ lit l'entête d'une tranche et retourne sa taille
%% suivit de la liste des champs. L'appel \ml+parse_crlf chan+ lit
%% %
%% \begin{listingcodefile}{wgetlib.ml}
%% open Http;;

%% type connection =
%%   { in_chan : in_channel; out_chan : out_channel;
%%     mutable on : bool; host : string; }

%% type header =
%%   { status : int; message : string; fields : (string * string) list }

%% type response =
%%   { connection : connection; header : header };;

%% let read_response connection =
%%   let (c, mes), lines = parse_response connection.in_chan in
%%   { connection = connection;
%%     header = { status = c; message = mes; fields = lines; };
%%   };;
%% \end{listingcodefile}
%% %
%% La copie du corps de la réponse doit être modifiée pour s'arrêter sur
%% la taille et non plus sur la fin de fichier.
%% %
%% \begin{listingcodefile}{wgetlib.ml}
%% let print_header chan header =
%%   let pr (x, y) =
%%     output_string chan (String.capitalize x); prerr_string ": ";
%%     output_string chan y;
%%     output_string chan "\r\n" in
%%   List.iter pr header;;
%% let error_report err mes reply =
%%   prerr_newline ();
%%   print_header Pervasives.stderr [err, mes]; prerr_newline ();
%%   print_header Pervasives.stderr reply.header.fields;
%%   error err mes;;
%% let print_chunk chan len =
%%   Printf.fprintf chan "%x\r\n" len;;
%% let print_status chan code mes =
%%   Printf.fprintf chan "%s %d %s\r\n" http_version code mes;;
%% \end{listingcodefile}
%% %
%% \begin{listingcodefile}{wgetlib.ml}
%% type echo = Everything | Body | Nothing
%% let response_OK = 200;;
%% let temp_redirect = 301;;
%% let perm_redirect = 302;;
%% let field name reply =
%%   try Some (List.assoc name reply.header.fields) with Not_found -> None
%% let content_length reply : int option =
%%   match field "content-length" reply with
%%     None -> None
%%   | Some s ->
%%       try Some (int_of_string s)
%%       with x -> error "Ill-formed field" "content-length";;

%% let moved reply =
%%   let s = reply.header.status in
%%   s = temp_redirect || s = perm_redirect;;
%% let location reply =
%%   field "location" reply;;
%% let chunked reply =
%%   match field "transfer-encoding" reply with
%%   | Some arg -> List.mem "chunked" (tokenize arg)
%%   | _ -> false;;

%% let buffer_size = 2048;;
%% let buffer = String.create buffer_size;;
%% let rec copy_region echo chan_in chan_out len =
%%   if len > 0 then
%%     let n = input chan_in buffer 0 (min len buffer_size) in
%%     if echo <> Nothing then output chan_out buffer 0 n;
%%     if n > 0 then copy_region echo chan_in chan_out (len -n)
%%     else error "End_of_file" "encountered prematurately";;

%% let rec copy_chunked echo chan_in chan_out =
%%   let len = parse_chunk chan_in in
%%   if echo = Everything then print_chunk chan_out len;
%%   if len > 0 then begin
%%     copy_region echo chan_in chan_out len;
%%     parse_crlf chan_in;
%%     if echo = Everything then output_string chan_out "\r\n";
%%     copy_chunked echo chan_in chan_out
%%   end else
%%     let h = parse_header chan_in in
%%     if echo = Everything then print_header chan_out h;;

%% let transfer echo reply out =
%%   match content_length reply with
%%     Some len ->
%%       copy_region echo reply.connection.in_chan out len
%%   | None when chunked reply ->
%%       copy_chunked echo reply.connection.in_chan out
%%   | None -> ();;

%% let dir_mode = 0o777;;
%% open Filename
%% let mkpath p =
%%   let normal_path =
%%     if basename p = "" then dirname p else p in
%%   let path_to_dir = dirname normal_path in
%%   let rec make p =
%%     try ignore (stat p)
%%     with Unix_error (ENOENT, _, _) ->
%%       if p = current_dir_name then ()
%%       else if p = parent_dir_name then
%%         error "Ill formed path" "contains \"..\""
%%       else begin
%%         make (dirname p);
%%         mkdir p dir_mode
%%       end in
%%   make path_to_dir;;
%% let unwind_protect f x g y =   try f x; g y with x -> g y; raise x;;

%% let regular path =
%%   try (stat path).st_kind = S_REG with Not_found -> false;;
%% let directory path =
%%   try (stat path).st_kind = S_DIR with Not_found -> false;;

%% let transfer_to_file connection file =
%%   let file =
%%     if basename file = "" then concat file "index.html" else file in
%%   mkpath file;
%%   Printf.fprintf Pervasives.stderr " ==> %s\n" file;
%%   let chan_out = Pervasives.open_out file in
%%   unwind_protect (transfer Body connection) chan_out
%%     close_out chan_out;;

%% let close_connection c =
%%   if c.on then begin shutdown_connection c.in_chan; c.on <- false end;;

%% let rec wget_url connection url =
%%   Printf.fprintf Pervasives.stderr "%s" url; flush Pervasives.stderr;
%%   let (hostname, port), path = parse_url url in
%%   let connection =
%%     match connection with
%%       Some c when c.on && c.host = hostname -> c
%%     | Some c when close_connection c; false -> assert false
%%     | _ ->
%%         let hostaddr =
%%           try inet_addr_of_string hostname
%%           with Failure _ -> (gethostbyname hostname).h_addr_list.(0) in
%%         let reply, request = open_connection (ADDR_INET (hostaddr, port)) in
%%         { in_chan = reply; out_chan = request;
%%           on = true; host = hostname; }
%%   in
%%   let get file =
%%     send_get connection.out_chan hostname file;
%%     let reply = read_response connection in
%%     if reply.header.status = response_OK then
%%       begin
%%         transfer_to_file reply (hostname ^ file);
%%         Some connection
%%       end
%%     else if moved reply then
%%       match location reply with
%%         Some new_url ->
%%           transfer Nothing reply Pervasives.stdout;
%%           Printf.fprintf Pervasives.stderr " --> %s\n" new_url;
%%           wget_url (Some connection) new_url
%%       | None ->
%%           error_report "Ill formed answer" "missing location" reply
%%     else error_report (string_of_int reply.header.status)
%%         reply.header.message reply in
%%   handle_syntax_error get path;;
%% \end{listingcodefile}
%% \begin{listingcodefile}{wget.ml}
%% open Sys
%% open Unix
%% open Url
%% open Wgetlib
%% let wget () =
%%   let argn = Array.length Sys.argv in
%%   if argn > 1 then
%%     let argv = Array.sub Sys.argv 1 (argn - 1) in
%%     match Array.fold_left wget_url None argv with
%%       Some c -> close_connection c
%%     | None -> ();;

%% handle_unix_error (handle_error wget) ();;
%% \end{listingcodefile}
%% \end{answer}
